Method and arrangement for detecting the spatial form of an object

ABSTRACT

The invention relates to a method of detecting the 3D shape of objects, in which the object is pressed into a plastically deformable mass provided with markers adapted to be evaluated photogrammetrically. After removal of the object from the marked mass, a plurality of photogrammetric images are taken from different views, and the images are evaluated photogrammetrically, the 3D shape of the object being calculated using an association of the markers corresponding in the images. The invention further relates to a corresponding arrangement for carrying out the method and is particularly suited for the detection of the 3D shape of objects whose 3D shape changes under load. One example of this would be the measurement of human feet.

The invention relates to a method and an arrangement for the detectionof the 3D shape of an object such as a human foot.

Detection of the three-dimensional shape (3D shape) of bodies and bodyparts is an important preliminary stage for the production and selectionof products whose shape needs to be adapted to the shape of the bodiesor body parts. Examples of such products are shoes, footbeds or archsupports, prostheses, ortheses, or articles of clothing.

It is known to detect body parts, e.g., optically, using lasertriangulation methods, methods based on stripe projection, or by opticaldetection of a sequence of silhouette outlines. In contrast to thesemethods, which are relatively involved because of their mechanically andoptically accurately constructed and calibrated systems, European PatentEP 0 760 622, “Sensing Process and Arrangement for the Three-DimensionalShape in Space of Bodies or Body Parts”, discloses a very simplesolution which is based on the photogrammetric evaluation of images ofthe body part which are obtained from non-calibrated imaging positions.To this end, the body part is clad in an elastic envelope provided withmarkers adapted to be evaluated photogrammetrically.

German Patent Application No. 100 25 922.7 “Automatischephotogrammetrische Digitalisierung von Körpern und Objekten” (Automaticphotogrammetric digitization of bodies and objects) further discloseshow by a special marking of the background of such markers, i.e. byapplying area markers, a referencing of the markers (point markers),i.e. an association of pairs of identical point markers in the differentoverlapping images can be facilitated and automated.

All of these aforementioned processes digitize the body when unloaded.In particular in the case of soft tissue, however, the 3D shape of aloaded body differs greatly from the 3D shape of an unloaded body.

For numerous products intended to be adapted to the body, such as afootbed, a seat shaped to fit a person's anatomy, etc., the spatialcoordinates of the body part deformed under load are required ratherthan those of the unloaded and non-deformed body part.

When measuring the sole of the foot for manufacturing a fitting insoleor arch support or a footbed made to fit, for example, the conventionalprocedure involves the person placing the foot into a plasticallydeformable mass, and obtaining an impression from this deformed massusing traditional molding techniques, such impression constituting thestarting position for preparing the arch support. This is an awkward,lengthy and costly process which could be considerably improved by anautomatic digitization of the sole of the foot under load.

It is therefore the object of the invention to provide a simple methodand a simple arrangement for the detection of the 3D shape of an object,which solve the problems explained above appearing in previous methodsand arrangements and which are more particularly suitable for simpledetection of the 3D shape of objects that need to be measured underload.

This object is attained by a method of detecting the 3D shape of anobject, wherein a plastically deformable mass is provided which has atleast one surface portion provided with markers adapted to be evaluatedphotogrammetrically, the object is impressed on the surface portion intothe mass such that the mass deforms corresponding to the 3D shape of theobject, the object is removed from the deformed mass, a plurality ofimages are taken of the deformed mass provided with the markers fromrespective different views, and the 3D shape of the object is determinedfrom the images by means of a photogrammetric process.

The object is further attained by an arrangement for detecting the 3Dshape of an object by photogrammetry, comprising an imaging system forobtaining photogrammetric images and a system for photogrammetricallyevaluating the images and for determining the 3D shape, which ischaracterized in that it further comprises a plastically deformable masswhich has at least one surface portion provided with markers adapted tobe evaluated photogrammetrically.

In accordance with the invention the optical detection of the 3D shapeof an impression material loaded and deformed by a body part is achievedin that the deformable impression material is characterized byhigh-contrast markers adapted to be evaluated photogranunetrically. Inparticular, in accordance with the invention these markings may consistof inclusions whose color differs from the color of the deformableimpression mass or of differently colored local colorations of theimpression material. By detecting a series of overlappingtwo-dimensional images (2D images) from unknown imaging positions of thedeformed and marked impression mass, the three-dimensional shape (3Dshape) of the impression and, hence, the 3D shape of the body partdeformed under load can be determined from these 2D images in a simpleway using known methods of photogrammetry.

Advantageous further developments of the invention are characterized inthe dependent claims.

Further features and advantages of the invention will be apparent fromthe following description of an embodiment with reference to thedrawings, in which:

FIG. 1 shows a plastically deformable impression mass used in oneembodiment of the invention and provided with markers adapted to beevaluated photogrammetrically, the impression mass being located in animpression frame and being used to prepare an impression of the sole ofthe foot under load;

FIG. 2 illustrates a schematic arrangement of cameras the positions ofwhich are not known and which are used to take overlapping 2D images ofthe impression which contain the markers and the positions thereof inrelation to the 2D coordinates of these images; and

FIG. 3 shows, as an example, a pseudo 3D representation of markersreproducing the three-dimensional shape of the footbed resulting fromthe impression in FIG. 2.

The method of the invention will now be described by way of example inconnection with a typical application, the detection of the sole of afoot under load for manufacturing a footbed that fits anatomically.

This example is, of course, by no means the only possible application ofthe method in accordance with the invention. For instance, the detectionof a 3D impression of the buttocks for manufacturing anatomicallyadapted seating surfaces would be another example where the detection ofthe 3D geometry of a body deformed by load is important. Many otherapplications are conceivable.

Within the scope of the idea of the invention, the terms “object” or“body” are by no means limited to human or animal bodies only, butfurther cover any physical bodies such as, e.g., mechanical models,antique statues, sculptures or similar three-dimensional spatial shapes.

As shown in FIG. 1, within the frame of a mold 1 there is held aplastically deformable mass 2 into which a person places his/her foot 3under load. At least a portion 7 of the surface of the mass 2 isprovided with markers 4 that are adapted to be evaluatedphotogrammetrically.

The mass 2 consists of a base material of a single color, into which anumber of markers 4 adapted to be evaluated photogrammetrically are letin, whose contrasting color stands out against the color of the basematerial. These markers may consist of embedded pins of a differentcolor, for example, which are made of the same material as the basematerial and are let into the base material. The pins may becylindrical, for example. The base material may consist of commonlyavailable deformable materials, for example, such as plaster, wax,alginate, or elastomers (e.g., silicones).

In a preferred embodiment a plastically deformable material is selectedwhich may be reversibly deformed; after deformation caused by thepressure exerted by the object and after termination of thephotogrammetric evaluation of the impression, the material may bereturned to its original form under the influence of energy, heat,light, or mechanical energy. Those of ordinary skill in the art arefamiliar with prior art materials of this type. When such a material isused, the mass provided with photogrammetric markers may be reused asoften as desired.

According to the invention, for a simple photogrammetric evaluation thebackground around these markers may also be marked by zones of adifferent color. This results in a marking on two levels, namelyconsisting of point markers and area markers, each of the area markerscomprising a plurality of point markers, forming a background of thepoint markers, and having a characteristic optical configuration.

The impression mass 2 will deform under the load of the foot placed ontothe surface portion 7 of the mass 2. The markers 4 which are visiblefrom outside will be shifted accordingly both laterally and in regard tothe depth thereof.

After the foot 3 has been removed from the mass 2, images are taken ofthe impression 6 that is left, illustrated in FIG. 2, from a number ofoverlapping imaging positions, using an imaging system which may includea camera or, as shown in FIG. 2, a plurality of cameras 5, and a numberof 2D images are prepared, in which the markers 4 occupy different XYpositions.

A person of ordinary skill in the art of photogrammetry and short-rangephotogrammetry knows how to evaluate such images for obtaining the 3Dcoordinates, which is why this will not be explained in greater detailherein.

In the photogrammetric method, first an image processing of the imagestaken is carried out, in which the markers respectively corresponding inthe images are associated with one another. Based on the markerassociation, the 3D shape of the impression and, from this, the 3D shapeof the object is then determined using known mathematical methods ofphotogrammetry.

Where point and area markers are employed, as set forth above, in theimage processing of the images first the area markers respectivelycorresponding in the images are associated with one another using theircharacteristic optical configurations. Then the point markers comprisedby the area markers and respectively corresponding in the images areassociated with one another with the aid of the area marker association.Then, using the point marker association, the 3D shape of the object isdetermined by means of a photogrammetric evaluation process.

In a preferred embodiment, a measuring rod may be mounted on that sideof the mold holding the plastically deformable mass which is opposite tothe photogrammetric imaging system, the measuring rod having arrangedthereon at least two photogrammetric markers at a known distance fromeach other. Such a measuring rod may consist, e.g., of a rigid, straightrod or plate having the markers applied thereon. The measuring rod mayof course also be arranged separately beside the mold. Thephotogrammetric images are then taken in such a way that at least twoimages of the impression bearing the markers are taken from any desiredpositions, with the measuring rod being also included in the images atthe same time. This allows the 3D shape of the impression or of theobject impressed to be calculated later in absolute values with the aidof the unit of length given by the measuring rod when the images areevaluated photogrammetrically.

FIG. 3 shows, by way of example, the pseudo 3D representation of themarkers reflecting the 3D shape of the footbed and thus permitting anautomatic production of a footbed using, e.g., a triaxial cutter.

In accordance with the invention, the impression material may also bemarked so as to make it suitable for photogrammetric evaluation in thatit is covered, prior to being loaded, by a marked elastic envelope, asdescribed in EP 0 760 622, which may consist of a textile material or asynthetic material, for instance. Preferably, this envelope is intendedto adhere to the impression material, so that the deformation of themass is transferred as accurately as possible to the deformation of theelastic envelope. Use of such an envelope solves the problem of hygienethat is created when there is a direct skin contact with the impressionmass.

As a disposable article, this envelope obviates the necessity of acostly disinfection of the impression material following each use.

The method according to the invention may be transferred to many otherimpression or molding applications, of which only a few will be listedas an example:

-   -   a) the manufacture of anatomically fitting seating surfaces by        use of a marked, flat, plastic mass into which the customer sits        himself down, thus leaving an impression of his buttocks        deformed by load;    -   b) the manufacture of spectacle frames that fit well in the nose        area by an impression of the region of the nasal root, using an        impression material marked in accordance with the invention;    -   c) the manufacture of copies of antique objects by producing an        impression in a plastic material marked in accordance with the        invention.

The terms “detection of the 3D shape of an object” and “determination ofthe 3D shape of an object” as used in the present specification and inthe claims are intended to be construed in a sense so as to includedetection of the negative shape of the object, i.e. the impression of anobject, since the positive shape of the object also implicitly resultstherefrom, i.e. as a result, the measurements of the object areprovided. These terms are further intended to relate also to thedetection of the 3D shape of a part of an object, such as, e.g., of afoot as part of the human body.

1. A method of detecting the 3D shape of an object, wherein aplastically deformable mass is provided which has at least one surfaceportion provided with markers adapted to be evaluatedphotogrammetrically, the object is impressed on the surface portion intothe mass such that the mass deforms under load of the object and animpression is formed in the deformed mass corresponding to the 3D shapeof the object such that the markers are shifted spatially with respectto their positions in the mass prior to deformation and with regard tothe depth of the impression, the object is removed from the deformedmass, a plurality of images are taken of the shifted markers provided onthe impression formed in the deformed mass from respective differentviews, and the 3D shape of the object is determined from the images bymeans of a photogrammetric process.
 2. The method as claimed in claim 1,wherein in the photogrammetric process an image processing of the imagesis performed, in which the markers respectively corresponding in theimages are associated with one another, and the 3D shape of the objectis determined using the marker association.
 3. The method as claimed inclaim 1 or 2, wherein the provision of the plastically deformable masswhich has at least one surface portion provided with markers adapted tobe evaluated photogrammetrically is effected by providing a plasticallydeformable mass having bodies enclosed therein which have a color thatis different from the color of the mass and which are visible fromoutside in the area of the surface portion.
 4. The method as claimed inclaim 1 or 2 wherein the provision of the plastically deformable masswhich has at least one surface portion provided with markers adapted tobe evaluated photogrammetrically is effected by applying markers of acolor that is different from the color of the mass onto the surfaceportion portion of the plastically deformable mass.
 5. The method asclaimed in claim 1 or 2 wherein the provision of the plasticallydeformable mass which has at least one surface portion provided withmarkers adapted to be evaluated photogrammetrically is effected byplacing an elastic envelope provided with markers on the surface portionof the plastically deformable mass.
 6. The method as claimed in claim 1,wherein the markers consist of point markers and area markers, each ofthe area markers comprising a plurality of point markers, forming abackground of the point markers, and having a characteristic opticalconfiguration.
 7. The method as claimed in claim 6, wherein in the imageprocessing of the images first the area markers respectivelycorresponding in the images are associated with one another using theircharacteristic optical configurations, and then the point markerscomprised by the area markers and respectively corresponding in theimages are associated with one another with the aid of the area markerassociation, and using the point marker association, the 3D shape of theobject is determined by means of a photogrammetric evaluation process.8. The method as claimed in claim 1, wherein the plastically deformablemass is provided in a mold which serves to hold the mass.
 9. The methodas claimed in claim 1, wherein the object is a body part of a person,whose 3D shape is to be measured under load.
 10. The method as claimedin claim 9, wherein the object is a foot.
 11. The method as claimed inclaim 1, wherein the object is a statue.
 12. The method as claimed inclaim 1, wherein the plastically deformable mass provided is areversibly deformable mass which, once the images have been taken, isreturned to its original form which it had taken prior to the impressionby the object.
 13. The method as claimed in claim 12, wherein thereversibly deformable mass is returned to its original form under theinfluence of energy, heat, light, or mechanical energy.
 14. The methodas claimed in claim 1, wherein prior to obtaining the images, ameasuring rod is arranged beside the surface portion of the deformablemass bearing the markers adapted to be evaluated photogrammetrically,the measuring rod having arranged thereon at least two additionalmarkers which are also adapted to be evaluated photogrammetrically, saidadditional markers being arranged at a known distance from each other.15. The method as claimed in claim 14, wherein at least two of theimages of the mass provided with markers are taken such that themeasuring rod is included in the image taken.
 16. An arrangement fordetecting the 3D shape of an object by photogrammetry, comprising animaging system for obtaining photogrammetric images and a system forphotogrammetrically evaluating the images and for determining the 3Dshape of the object, wherein the arrangement further comprises aplastically deformable mass operable to deform under load of the objectsuch that an impression is formed in the deformed mass corresponding tothe 3D shape of the object which has at least one surface portionprovided with markers adapted to be evaluated photogrammetrically, saidmarkers being configured to shift spatially with respect to theirpositions in the mass prior to deformation and with regard to the depthof the impression when the mass deforms under load of the object suchthat said images are of the shifted markers provided on the impressionformed in the deformed mass.
 17. The arrangement as claimed in claim 16,wherein bodies of a color different from the color of the mass areprovided to serve as markers, the bodies being enclosed in the mass andbeing visible from outside.
 18. The arrangement as claimed in claim 16,wherein color markings serving as markers are provided on the surfaceportion of the mass and have a color that is different from the color ofthe mass.
 19. The arrangement as claimed in claim 16, wherein an elasticenvelope is provided having markers applied thereon, which is pulledover the mass in the area of the surface portion and adheres to themass.
 20. The arrangement as claimed in any of claims 16 to 19, whereinfurther provided is a mold to hold the mass.
 21. The arrangement asclaimed in claim 16, wherein the platically deformable mass is areversibly deformable mass.
 22. The arrangement as claimed in claim 21,wherein the reversibly deformable mass is made such that it can bereturned to its original form under the influence of energy, heat,light, or mechanical energy.
 23. The arrangement as claimed in claim 16,wherein a measuring rod is provided which bears at least two additionalmarkers adapted to be evaluated photogrammetrically, arranged at a knowndistance from each other, the measuring rod being arranged beside thesurface portion of the deformable mass.
 24. The arrangement as claimedin claim 23, wherein the measuring rod is connected to the mold holdingthe mass.